Skip to content

nvidia/jetson-orin-nano-super: Enablement for GPU acceleration#23

Merged
blitz merged 30 commits into
cyberus-ctrl-os:mainfrom
samueldr-at-cyberus:feature/hw/jetson-orin-nano-super-3d-accel
Feb 27, 2026
Merged

nvidia/jetson-orin-nano-super: Enablement for GPU acceleration#23
blitz merged 30 commits into
cyberus-ctrl-os:mainfrom
samueldr-at-cyberus:feature/hw/jetson-orin-nano-super-3d-accel

Conversation

@samueldr-at-cyberus

@samueldr-at-cyberus samueldr-at-cyberus commented Feb 13, 2026

Copy link
Copy Markdown
Contributor

What works:

  • Proprietary HW acceleration under mainline linux v6.12
    • Under X11 with glx and egl support
    • Wayland compositors can be used, using modesetting and acceleration

What needs more investigative work:

  • gbm platform under eglinfo fails
  • Wayland clients (or wayland platform under eglinfo) end-up using llvmipe software rendering

These screnshots were taken using a capture card, from an X11 session:

image image

This is YMMV

All of the pain setting up the display driver is likely caused by their driver stack being barely able to work in their very narrow supported use-case. The moment drivers load in a different order, or are configured differently, all bets are off about whether or not it's going to work.

There are two big issues here:

  • Inability to take over the display peripherals
  • Problematic setup for card0 usage

Inability to take over the display peripherals

(Open for the details)

When the platform firmware leaves a simple-framebuffer compatible, the driver stack is either unable to take over (with simpledrm) or relies on older specific behaviours related to legacy fbdev devices. An efifb should also break in the same manner.

This is also upheld by the fact that manually doing the work (e.g. unbinding the simpledrm driver, as we do here) can make the driver work.

Problematic setup for card0 usage

(Open for the details)

This is less of a direct problem, and more of a consequence of their setup.

This is not an issue for X11 as the proprietary X11 driver seems to implement bespoke logic to pick the correct card when tegra-drm is loaded.

Wayland compositors and other DRM users like to use /dev/dri/card0 as the default, or sometimes relatively hardcoded card to use.

Since the nvidia-drm driver (apparently) needs to have tegra-drm loaded first, and since tegra-drm has its own minor assigned, which will be the first available, the tegra-drm driver will have the card0 device, while nvidia-drm will have the card1 identifier.

This is worked around when simpledrm is present in our systems by making tegra-drm load while card0 is active. This means that tegra-drm uses card1, and when simpledrm is dropped the card0 device is freed for nvidia-drm.

This could be improved by forcing a DRM driver built for the purpose of taking card0 to be loaded, instead of relying on simpledrm to be present.

This is not strictly an issue with nvidia-drm, but an issue with the broader DRM-using ecosystem, where there is no common way to point to a specific device to be used when it's not card0. Thus, there is no proper way to point to /dev/dri/by-path in a generic manner, if pointing to a path is even supported.

@samueldr-at-cyberus samueldr-at-cyberus force-pushed the feature/hw/jetson-orin-nano-super-3d-accel branch 2 times, most recently from 2bef2a4 to 42e48b2 Compare February 16, 2026 14:42
Comment thread modules/hardware/devices/nvidia/jetson-orin-nano-super/default.nix Outdated
@samueldr-at-cyberus samueldr-at-cyberus force-pushed the feature/hw/jetson-orin-nano-super-3d-accel branch 2 times, most recently from d887b4c to b8178d3 Compare February 20, 2026 15:52
@samueldr-at-cyberus

Copy link
Copy Markdown
Contributor Author

This last push tries to be less clever about how the drivers are being loaded, which it turns out is needed, since the reason it was working (somehow always) on my system was likely due to a race condition. When loading the same system configuration from the NVMe drive compared to a "usb installer iso" build, the latter wouldn't work, and I'd be reproducibly hitting the same issue my colleagues were.

@samueldr-at-cyberus samueldr-at-cyberus force-pushed the feature/hw/jetson-orin-nano-super-3d-accel branch 4 times, most recently from 55d4959 to 7335aed Compare February 20, 2026 22:48
See the next commit.
This will be used to make the style check only apply to the locked
Nixpkgs version.
It's redundant because at the `flake.nix` level this was already being
handled.

It's wrong because it's using the `lib` from the `flake-parts` input,
which ***is not*** the `lib` from the `nixpkgs` or `pkgs` input!

So whatever override we were passing for `nixpkgs` wouldn't affect that
`lib`, and thus the `version` on the `lib` was whatever the
`flake-parts`'s Nixpkgs was at.
This is one way Flakes let us down: there is no way to make the `check`
run only when inputs were not overridden, or additional ways to affect
the evaluation.

So, just like Sisyphus, we need to push the version number up every time
the formatting rules change in the formatter that is on the
`nixos-unstable` release we track, compared to whichever versions we run
as additional CI checks.

Instead of handling this manually, let's just look at the locked
version number from the release channel URL.
This unshackles the evaluation of the package from the NixOS evaluation.
Turns out we already basically were at r36.5, and this will have been
the cause of some of the issues getting the GPU drivers going.
This is *basically* the open GPU driver codebase, but in a special Tegra
configuration.

It also is a different branch, with a version number that may not be
matching anything equivalent on the open GPU drivers.

This is not sufficient to have hardware acceleration. The proprietary
drivers also have to be used.

The proprietary driver version *must* match the version of the this
display driver, or else things may not work, or worse.
…nel-display-driver

This was backported as it looked like it may be able to fix the GPU
driver usage with mainline 6.12. It did not.

This did fix the `fbdev` parameter for 6.12.
This is not forced on end-users.

This is provided because it *may* be needed to better support all
features of the platform. Even with OOT drivers, some features won't
work with mainline, or worse.
The `nvidia-l4t` package includes the proprietary libraries, the ones we
don't have the equivalent sources to.

These libraries are where the GPU acceleration lives.
@samueldr-at-cyberus samueldr-at-cyberus force-pushed the feature/hw/jetson-orin-nano-super-3d-accel branch 2 times, most recently from 98ac555 to 578f17e Compare February 25, 2026 21:01
This is the modern convention on Linux systems. Access to device nodes
is mediated through ACLs being given by the login manager (not the
display manager).

This ensures common NixOS module configuration *just works* without
needing to change the groups of greeters or compositors to include
`video`.

This fixes "just" using gdm and gnome (when `card0` is `nvidia-drm`).

Also fix the `mknod` path at the same time.
Also introduce `enableStage1KernelModules` to allow end-users the option
to disable the modules.

This *should* generally be done for options that are lists, as those are
poorly composable, and removing an element from a list in the modules
system is generally hard.
When users otherwise don't explicitly care, use 6.12, as `nvidia-oot`
only claims to support up to 6.15, and 6.12 is the prior LTS.

This only sets the kernel using `mkDefault`, so this will not affect any
selection users have made for themselves.
This should end-up being accurate when the modules needing it are
enabled, but just in case, make it so `/run/opengl-driver` will be
automatically enabled when hardware acceleration is enabled.
This is used to solve the issue where the NVIDIA drivers are (still)
unable to take control of the device when they are used by a DRM driver.

What's funny here is that this makes the graphics driver work on
mainline 6.12, whereas when using `FBDEV` in the kernel config, starting
xorg would reset the board.
@samueldr-at-cyberus samueldr-at-cyberus force-pushed the feature/hw/jetson-orin-nano-super-3d-accel branch from 578f17e to 9e114c5 Compare February 25, 2026 22:24
…uffer and enabling hardware acceleration

This should help end-users who might be disabling that quirk to figure
out issues, and leaving it accidentally enabled.
…ules...

This will be managed in the follow-up commit. Refer to it for the
rationale
…ever way

The clever way (trying to “react” on system-wide changes) ended-up
having a race condition that could be triggered in unclear
circumstances... and that race condition *was apparently why it
worked*.

So this rework instead implements that race condition as a feature,
instead of relying on my good luck.

This is still implemented as a `udev` rule as it is the easiest and most
robust method to implement the required dance here:

 - Load `tegra-drm` such that it takes `card1`
 - Unbind `simpledrm` such that it frees `card0`
 - Load `nvidia-drm` such that it takes `card0`

This, it turns out, is the secret sauce to make wayland compositors
*just work* with the L4T drivers.

The load order of the modules was already known to matter a lot: the
`nvidia-drm` driver *must* be loaded after `tegra-drm`. And yes, the
`tegra-drm` module *needs* to be loaded for it to work.

The real difference compared to before is that having `card0` taken by
the `nvidia-drm` driver means that, without some special proprietary
sauce like in the X11 L4T driver or manual configuration, the software
using DRM *should* pick the appropriate card by default.

This matters when some compositors don't really expose a good way to
pick a given `card` or `render` node.

In an ideal world, either the `tegra-drm` would have its devices
unregistered when `nvidia-drm` is loaded, or we could affect the
allocation of the `minor_str` of the DRM devices so that a given device
is prefered, or `nvidia-drm` could be loaded first without breaking.

Alas, we're beholden to the drivers we are given.

This strategy *does not* work when `simpledrm` is not in use. As of this
commit, the use of the vendor kernel as-configured without `simpledrm`
wouldn't work as there is nothing tying the `tegra-drm`+`nvidia-drm`
driver loading to the classic `fbdev` driver. Such systems were *anyway*
broken in the same way previously.
While this is not supported to *build*, we have CI checks against 24.05,
so let's make CI work.
Let's not rely on `fbdev`, even though *it nominally works* for the
vendor kernel. Both the vendor kernel and mainline should nominally work
like the other for the system integration. And that means that it should
enable the `tegra-drm` and `nvidia-drm` modules with the same semantics.

When using the `fbdev` setup, the `tegra-drm` and `nvidia-drm` *are*
effectively loaded (despite the blacklist?) but are loaded in a way
where `card0` is the `tegra-drm` device, which would be a notable
difference for non-X11 graphical use-cases.
…module

Turns out the stack is ***very*** fussy, and there is *something* that
can happen while loading the drivers, if it happens while the system is
kinda busy, and in concurrence with a lot of other things.

The reason this is assumed to be some race condition or similar
situation is that the more things are made to go slowly, for example
with sleep or by adding dynamic debug prints to the kernel, the more
likely things are to work.

So let's stop being clever and reactive about the environment. Instead,
let's just brute-force our way to a working display output, and make the
whole world wait for the driver to load. But only when the graphical
target is involved.

I believe this is something that the vendor is having difficulties with,
as other distributions require turning of “SOC Display Hand-off” in the
platform firmware for accelerated display support, and their newer
Jetpack platform, for AGX Thor, also requires it for the vendor system.
The fact that the vendor ships their kernel with fbdev also points
toward that being true.
@samueldr-at-cyberus samueldr-at-cyberus force-pushed the feature/hw/jetson-orin-nano-super-3d-accel branch from 9e114c5 to 1e011a5 Compare February 26, 2026 02:20
@blitz

blitz commented Feb 26, 2026

Copy link
Copy Markdown
Contributor

The recent changes don't fix the doom loop of fabric errors for me. For posterity:

  • without any display attached -> driver attaches, system comes up
  • with DP display attached (tested with LG and HP displays) -> doom loop of fabric errors and kernel warnings (see attached logs)

jetson-orin-nano-with-dp-dell-display.log
jetson-orin-nano-with-dp-lg-display.log

@blitz blitz left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have to leave debugging this for another day. From my end this can be merged.

@blitz blitz added this pull request to the merge queue Feb 27, 2026
Merged via the queue into cyberus-ctrl-os:main with commit 1988b46 Feb 27, 2026
6 of 9 checks passed
@samueldr-at-cyberus samueldr-at-cyberus deleted the feature/hw/jetson-orin-nano-super-3d-accel branch February 27, 2026 14:33
@samueldr-at-cyberus

Copy link
Copy Markdown
Contributor Author

While doing final checks for reproducing the issue @blitz was facing, I discovered that on my local testing setup, using the DisplayPort port on a display reproduced the failure, while using the HDMI port on the same display did not.

All my prior testing was done on an HDMI USB capture card.

So if enabling the driver fails to enable the display, and keeps the device basically locked-up via continuous prints on the (serial) console, try an HDMI input. YMMV.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants